Learning Structure with the Trace Norm Distribution
نویسندگان
چکیده
We consider the problem of learning model structure. We assume that data is generated by one or more trace norm distributions [3]. We find that, as with other unsupervised problems, there are a number of equallycorrrect solutions. Which solution is best depends on the interpretation of distances in the data space. 1 Trace Norm Distribution Let X ∈ R. The trace norm distribution is defined as P λ (X) = 1 Z λ exp(−λ‖X‖Σ), (1) where the superscript is not exponentiation, but rather a designation that the distribution is specific to the matrix size. ‖X‖Σ is the trace norm of X (the sum of singular values of X), and Z λ = ∫
منابع مشابه
Learning with the weighted trace-norm under arbitrary sampling distributions
We provide rigorous guarantees on learning with the weighted trace-norm under arbitrarysampling distributions. We show that the standard weighted trace-norm might fail when thesampling distribution is not a product distribution (i.e. when row and column indexes arenot selected independently), present a corrected variant for which we establish strong learningguarantees, and demon...
متن کاملExcess risk bounds for multitask learning with trace norm regularization
Trace norm regularization is a popular method of multitask learning. We give excess risk bounds with explicit dependence on the number of tasks, the number of examples per task and properties of the data distribution. The bounds are independent of the dimension of the input space, which may be infinite as in the case of reproducing kernel Hilbert spaces. A byproduct of the proof are bounds on t...
متن کاملMatrix completion with the trace norm: learning, bounding, and transducing
Trace-norm regularization is a widely-used and successful approach for collaborative filtering and matrix completion. However, previous learning guarantees require strong assumptions, such as a uniform distribution over the matrix entries. In this paper, we bridge this gap by providing such guarantees, under much milder assumptions which correspond to matrix completion as performed in practice....
متن کاملCollaborative Filtering with the Trace Norm: Learning, Bounding, and Transducing
Trace-norm regularization is a widely-used and successful approach for collaborative filtering and matrix completion. However, its theoretical understanding is surprisingly weak, and despite previous attempts, there are no distribution-free, non-trivial learning guarantees currently known. In this paper, we bridge this gap by providing such guarantees, under mild assumptions which correspond to...
متن کاملMulti-Stage Multi-Task Learning with Reduced Rank
Multi-task learning (MTL) seeks to improve the generalization performance by sharing information among multiple tasks. Many existing MTL approaches aim to learn the lowrank structure on the weight matrix, which stores the model parameters of all tasks, to achieve task sharing, and as a consequence the trace norm regularization is widely used in the MTL literature. A major limitation of these ap...
متن کامل